| Review 1 | |
| PC member | Clayton Lewis |
| Overall evaluation |
Proposes a map of substrates projects that distinguishes shades of theoretical and practical
concerns, along one axis, and a focus on the past and present vs the future, on a second axis.
The authors offer this as a springboard for discussion, and it works as such. Here are some of
the questions it provokes from me: How is it judged how present-oriented, or future oriented an existing, implemented system is? Although Webstrates (I believe) exists and works in the here and now, its lead creator puts it in the future region of the proposed map. Does this signal aspirations not yet realized in the design? Or aspirations perhaps realized, but not fully evaluated against practical use, and hence modestly not yet claimed? The same question arises for Boxer, which is shown on the future-most edge of the map. It has existed for decades, and even has had periods of extensive use (while now getting a thorough rebuild and updating). Are there "latent" concepts in Boxer that the authors feel haven't been fully realized, but they expect to emerge in future? Would it be useful to distinguish different strains of theoretical work? Projects might aim to innovate in how concepts are presented to users, for example, or in how they are realized in an implementation. Some ideas about implementation would seem to be theoretical in character (though not all do). But the sorts of theory involved in these two kinds of work seem quite different. The map isn't so densely populated that this sort of distinction couldn't be marked; color could be a resource. Perhaps this thought overlaps with what the authors say about different kinds of research products (episteme, etc.) Could one imagine zooming in on the map, to see the particular research questions a project is addressing, and what mechanisms are being proposed and explored? This would be of particular interest if doing so revealed that there are some questions being addressed by more than one project, since that would set the stage for productive exchanges and comparisons. But differences in the questions would also be of interest, and productive of discussion: "I've identified issue X as of prime importance, and you're not working on it in your project. Why not? Do think it's been solved, or that it's impossible?" The emergence of AI tools raises questions about almost anything, these days, and some questions may be of particular interest for substrates work. Taking substrates to be motivated by changing the way(s) people can create and interact with software, do LLMs offer something new? Might they ease some pain points that would then require less reform? Do they offer new ways of dealing with user problems (cross reference here to the Krebs submission)? Could a reliable software system be assembled from vague and natural user expressions, translated by AI into tight, executable code, and could a substrate have this character? This might create a system that's more readable for humans than those expressed directly in code. |
| Review 2 | |
| PC member | Tomas Petricek |
| Overall evaluation |
The paper makes an attempt to organize (in some preliminary way) some of the existing work
related to programming substrates. This is a very worthwhile aim - I fully agree that one of the
issues with our field is the lack of coherence, shared interest or shared goals. The "map" that
this paper provides is certainly one step towards relating some of the work and hopefully
enabling authors to better understand each other, speak to each other and collaborate. I think
this is a submission that will lead to invaluable discussion and it should be presented at the
workshop. There is a number of ideas that came to mind when reading the paper, which (I hope) may contribute to the discussion at the workshop. First, there is a number of terms that our community uses such as 'substrate', 'programming system', 'computational' or 'dynamic media'. While they are seem somehow related and possibly overlapping, it is not quite clear if any of them mean exactly the same thing or how they differ. This is possibly another dimension (and maybe not a moral one?) but it may be useful addition to the discussion inspired by this paper. I also would like to problematize a little bit the notion of "theory" and "practice" in the paper. In a conventional understanding, we can think of theory as mathematical theory (PLT) or more conceptual theory (analytical philosophy, perhaps) or user experience theory. But I'm very much convinced that certain kinds of practical implementations of an idea play the role of a theory too. To my shame, I never read the paper fully and properly, but Naur's "Programming as theory building" (1985) may be relevant here! Also, I would say that our "Evaluating programming systems design" (PPIG 2019) paper was similarly asking if (some suitable) form of programming system could be made equal to conventional theories. In other words, I think it may not always be clear what is theory and what is practice. Concretely, the authors talk about the Infusion system as fully in the theoretical realm - interestingly, this is something I would not expect. Based on the hallway demo I saw at <P>25 (and on the fluid.cell submission to the workshop), my impression always has been that the Infusion system is at least partly rooted in practical exploration - perhaps leading to more high-level theoretical insights (as in Gabriel's practice preceding theory). This makes me think of an interesting exercise for the workshop - show the audience the two dimensions and some of the systems that the paper currently draws on the chart and ask the audience to place them on the chart! It would be interesting how much agreement/disagreement this will lead to. Another work that is somewhat difficult to position (I think) is the work Yann Trividic on publishing (from Substrates'25) - it is very much rooted in the present (in that he is looking at systems that exist today rather than proposing some new substrate), but he is seeing them from a new perspective - you rethink what exists. The "potential" is not so much new technology/substrate, but a new perspective. Finally, the paper made me ask a question (which is perhaps the elephant in the room?) - why are we finding it so difficult to align the substrates research agenda in a coherent way? Many people in the community agree that there is some kind of common theme, but I think the paper (inadvertently) reveals that we do not quite know what it is! I do not think people working on "pure functional programming" in the early days of the paradigm - or computer scientists defining the Algol research programme - had to think so much about how to structure their research ideas. (Maybe their research agenda was simplistic or boring? But maybe there is something to learn from those successes...) |
| Review 3 | |
| PC member | Joel Jakubovic |
| Overall evaluation |
This submission is about how we can map the space of actual and possible substrates (whatever a
substrate may be!). It seizes on the fertile metaphor of "dimensions spanning spaces" and sows a
few tentative examples as discussion seeds: - Theory (science and philosophy) vs practice (engineering and iteration) - Existing (becomes a substrate retroactively) vs potential (designed on purpose to be a substrate) - Explanatory vs actuating vs generative theory I think the submission is a fine prompt for further discussion at the workshop, as it intends to be, so I'll save my thoughts on its questions for then. In the meantime, I do have some thoughts that I hope will be helpful to the authors in preparing for it. I also have one or two small revision requests, for the purpose of feedback on the writing. (They may read them as "if we were doing a revision process, this is what I'd ask for - take note for the presentation") I'm a little confused by the "existing vs potential" proto-dimension and its examples. Webstrates certainly exists, as does Lopecode - is the idea that WS was designed specifically to be a substrate, while Lopecode just ended up that way? A retro-fitting of existing Observable infrastructure? This dimension should be renamed according to the helpful paragraph about "reading substrate nature into something already produced". Ideas: "re-purposed vs designed", "improvised vs designed", "accidental substrate vs purposive substrate", "parasitic vs autotrophic"? Even "postmodern vs modernist"? One thing I strongly believe and warn for dimensions frameworks is to be careful not to classify the relevant population according to "good" and "bad" or terms that substitute for them. Not that this submission suffers from that - it's just something I'd like to emphasise. "Parasitic" or "modernist" should never just become synonyms for "stuff we don't like" - instead, we should be able to explain what we like and dislike *in terms of* the descriptive characteristics of how the things behave and what they encourage / discourage. The point is that when I consider introducing terms like "parasitic" or "modernist", I am doing so in full expectation that there will be some examples of each that we like and dislike, and I am trusting that researchers, if they adopt the terms, will maintain the same discipline. In light of this, I think the title "moral dimensions" is premature - the early stages of such a field should be focused on what "is", and we can look forward to fighting over what "ought" later! ;) I'm afraid I don't have a good replacement, but I think "moral" misleads (or prematurely commits) as to the true content of the submission and, even if it didn't, it's still too early to look for real moral dimensions anyhow. Just "dimensions" would feel appropriate to me at this early stage. Extending point-like examples into stretched ellipses is a very good move and I think it should be the default premise for dimensions frameworks. Very often a single named example (like "spreadsheets") is a convenient shorthand covering diversity that might be relevant to some concerns (which spreadsheet? If Excel - which Excel? Etc). A given property might apply to an example in some sense but arguably not in another sense, or only when it's used in a certain way, etc. So I agree, let's openly acknowledge the vagueness (and ambiguity, poly-semanticity) inherent in much of what we're studying. Perhaps a good question for the theory side of the field: what constitutes a point? If not "Excel 2025", then perhaps "The installed copy of Excel 2025 sitting on my computer"? Or even "This particular USE I made of Excel on such-a-date"? Questions like these are still unresolved from when I worked on Technical Dims of Programming Systems (I'll plug Section 7.1 of my thesis as a source of ideas which apply just as well to substrates dimensions - indeed I would really like to ask in the workshop what, if anything, distinguishes a substrate from a programming system!). A system was designed to support a family of tasks for some audience; its actual use may differ from what was intended; we're very often interested in the uses of the system rather than ... well, whatever's left of the system when we subtract its uses! (If anything.) Following Wittgenstein, if the meaning of a word is just how it is used, then this may be an interesting framing, but it only *hints* at any helpful ways to study word meanings - it's not clear how to enumerate the fractally complex set of "uses" of a word (or anything, eg a system or substrate) At some point I decided that we're really in the business of classifying something like software "organisms", so we can probably learn a lot from however naturalists manage their diversity of lifeforms and their commonalities. I am certain that, from prior work with such communities, the authors know more about that than me :) Framing the research as three of Aristotle's five Virtues of Thought is a welcome organisational move. Towards the end, I appreciate the point that theory and practice inform each other, following the Gabriel quote earlier, as a guard against unproductive blinkers on thinking. However, I don't really understand the "explanatory theory vs actuating theory" split. I ask the authors to better motivate the former idea of "explaining the behaviour of an existing system against a theoretical background". Why exactly do we want to to do this...? In science, we explain the behaviour of the world so as to predict what it's going to do next and to control it to be more like what we want. This applies to the "design" of biological systems because evolution didn't leave a helpful manual behind - but for substrates designed by people for specific things, can't we already explain their behavior either by reference to their explicit design, or by reference to the implementation (in the case of behavior that deviates from what the designer wanted)? When do substrates behave in a way that violates our expectations? If anything does need explaining this way, it might be the unintended or un-designed uses of such systems (that's mysterious Nature creeping in again). But I'm still struggling to motivate why we might want such an explanation. Perhaps in order to justify the continued support of a technology (like Flash) against some suit (Adobe) who thinks it's obsolete and nobody needs it (or that they should all just use something else)? Or perhaps the logic is that the more unpredictably versatile something is, the more users can adapt it to their needs, so by understanding what encourages unintended behaviour in existing substrates, we can use it to design malleable software? For the "actuating" theory, I appreciate the stack frame / reactive net example - but that only gives me the haziest idea of what the theory may be for! I eagerly await discussion on the day, but a little more detail for this available beforehand will get the authors better discussion. Some specific requests: "Basman's responses to the 6 principles" where? Please link or cite "and explain[s] how software systems can be built which remedy them" - remedy what? The principles? Unclear what this refers to. The whole sentence is unwieldy and would benefit from a rewrite; I struggle to grasp what it is trying to say. |